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Abstract. In collaborative robotic cells, a human operator and a robot 
share the workspace in order to execute a common job, consisting of 
a set of tasks. A proper allocation and scheduling of the tasks for the 
human and for the robot is crucial for achieving an efficient human- 
robot collaboration. In order to deal with the dynamic and unpredictable 
behavior of the human and for allowing the human and the robot to 
negotiate about the tasks to be executed, a two layers architecture for 
solving the task allocation and scheduling problem is proposed. The first 
layer optimally solves the task allocation problem considering nominal 
execution times. The second layer, which is reactive, adapts online the 
sequence of tasks to be executed by the robot considering deviations 
from the nominal behaviors and requests coming from the human and 
from robot. The proposed architecture is experimentally validated on a 
collaborative assembly job. 


Keywords: task allocation, task scheduling, human robot collaboration 


1 Introduction 


Collaborative robots are getting more and more common and appealing for in- 
dustrial applications (see e.g. [1}[2]). One of the reasons of this success is the 
possibility of creating a synergy between a human operator and a robot, two 
agents with complementary skills, for allowing the execution of jobs that nei- 
ther the robot nor the human alone can execute. In order to achieve such an 
ambitious goal it is necessary to enable the collaboration as much as possible 
by creating mutual awareness and communication between human and robot. A 
lot of work has been done in this direction for the execution of a specific task. 
In human motion prediction strategies have been developed for making 
the robot aware of the behavior of the human and to react accordingly. In Elle] 
the concept of legibility has been exploited for making the intentions of the 
robot clearly understandable by the human who can, therefore, react accord- 
ingly. In the authors exploit electromyography (EMG) signals to estimate 


2 Andrea Pupa et al. 


the human muscle fatigue. Subsequently, this information is used to adapt the 
robot behavior, improving the human-robot interaction. In {8| a verbal feedback 
strategy is implemented. The robot informs the human about what action it 
will take with verbal information (self-narrative feedback) and what action the 
human should perform (role-allocative feedback). Moreover the verbal feedback 
is also exploited to inform the human about the result of a task (empathetic 
feedback). Instead, in [o] the authors propose the use of a wearable device to 
recognize the movements of the human’s forearm. This recognition is then used 
as input to control a semi-autonomous quadrotor. 


In the industrial practice, collaborative cells are exploited for executing a job, 
composed by a set of tasks. Thus, it is important to understand how to allocate 
and schedule all the tasks to the human and to the robot (see e.g. [10)). 


The multi-agent task allocation problem has been widely studied in the in- 
dustrial scenario. In the authors implement an algorithm for solving the 
assembly line worker assignment and balancing problem (ALWABP). The idea 
is to find the tasks to be assigned to each work station, thus worker, in order 
to minimize the cycle time respecting the precedence constraints. In the 
authors use the Hungarian Algorithm to optimally assign a mission between the 
AGVs, taking into account the traffic state. 


The problem of task allocation has been addressed also for the human-robot 
collaboration in industrial scenarios. Several works formulate the task alloca- 
tion problem as an optimization problem where the human characteristics are 
encoded in the cost functions (see e.g. (13}{76}). The main problem with these 
approaches is the computational complexity, which makes them unsuitable for 
fast rescheduling. In an integer linear programming (ILP) problem for a 
collaborative assembly scenario is implemented. Due to the complexity of the 
problem, a metaheuristic approach is used to solve the optimization problem. 
In a genetic based revolutionary algorithm for real-time task allocation is 
proposed. The framework takes into account both parallel and sequential task, 
while it aims and minimizing the total makespan and the payment cost. In 
a framework based on two level of abstraction and allocation in HRC scenario 
is presented. The first layer is responsible of solving the Task Allocation prob- 
lem based on a cost function. The tasks are represented with an AND/OR graph 
and the optimal task allocation is obtained solving the A* algorithm. The second 
layer, instead, handles the task execution and the respective failures. However, 
if the system detects some errors, it is necessary to recalculate the optimal so- 
lution, which is a computationally demanding procedure. 


These task assignment approaches assume that the time necessary for execut- 
ing each task, either by the human or by the robot, is constant. This assumption 
is quite conservative and it is in sharp contrast with what happens in reality. 
In fact, it is quite unlikely that the human executes a given task in the same 
amount of time and this may lead to a strong inefficiency in the task allocation. 
Some works have addressed the variable task duration problem. In the au- 
thors solve the variable task duration presenting a framework composed by two 


A dynamic architecture for task assignment 3 


stages: the proactive scheduling stage and the reactive scheduling stage. In the 
first stage, the processing times are treated as Gaussian variables. Instead, the 
second stage is responsible of modifying the schedule to deal with uncertainties. 
In a scheduling architecture based on time Petri Nets is presented. The goal 
is to minimize the idle time of both human and the robot adapting the planned 
activities in a receding horizon. For this reason, the durations of the two agents 
are considered variable and fitted online with a Gaussian Mixture Model. How- 
ever, this frameworks cannot handle the task assignment problem, which must 
be pre-solved. Moreover, the human and the robot are treated as two separated 
entities, without considering interactions and communication. 


In order to fully enable a human-robot collaboration, awareness and com- 
munication should be implemented both at the task execution level and at the 
task planning and scheduling level. Thus, it is important to create awareness 
about the real duration of the tasks executed by the human in order to enable 
the scheduler to replan the assigned tasks in order to maximize the efficiency 
of the collaboration. Furthermore, human and robot must be able to interact 
through the scheduler in order to make their collaboration smoother and more 
efficient. The human, because of its specific experience and expertise, can decide 
to execute a task that is assigned to the robot and the robot can decide to as- 
sign to the human one of its tasks (e.g. because it fails the execution). All these 
decisions should be communicated through the task scheduler. 

In this paper we propose a novel framework for task assignment and schedul- 
ing for collaborative cells that is aware of the activity of the human and that 
allows the human and the robot to take decisions about the tasks they need to 
execute. The proposed framework is made up of two layers. Given a job to exe- 
cute, an offline task assignment layer allocates the tasks to be executed by the 
human and the ones to be executed by the robot by building a nominal schedule. 
The scheduler layer, according to the real execution time of the human operator 
and to the decisions taken online by the human and by the robot, reschedules 
the tasks, possibly overriding the decisions taken by the task assignment layer, 
in order to improve the efficiency of the execution. 

The main contribution of this paper are: 


— A novel adaptive framework for task assignment and scheduling that takes 
into account real execution time of the human and communication with 
human and robot for dynamic rescheduling 

— A strategy for dynamic rescheduling that is effective and computationally 
cheap, i.e. suitable for industrial applications, and that allows human and 
robot to communicate their needs to the scheduler. 


The paper is organized as follows: in Sec. [2] the task assignment and dynamic 
scheduling problem for a collaborative cell is detailed and in Sec. Jan optimiza- 
tion procedure for solving the task assignment problem is proposed. In Sec. 
an algorithm for dynamically scheduling the task of the robot and for allowing 
the human and the robot to communicate with the scheduler for changing the 
set of their assigned tasks is illustrated and in Sec. Blan experimental validation 
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of the proposed architecture is presented. Finally in Sec. [6]some conclusion and 
future work are addressed. 


2 Problem Statement 


Consider a collaborative cell, where two agents, a human operator H and a robot 
R, have to execute a job J together. The job can be split into a set of taskq!] 
(T,,..., Ty) and each task T; is associated with a nominal execution time t;(A), 
where A € {H, R} represents the agent that executes the task. 

The cell is endowed with a monitoring unit that, for a task T; assigned to the 
human, estimates online the execution time. Several strategies for implementing 
the monitoring are available in the literature: sequential interval networks (22], 
interaction probabilistic movement primitives 23], Open-Ended Dynamic Time 
Warping (OE-DTW) to name a few. 

We assume that the tasks are independent, i.e. T; and T}; can be executed in 
parallel for any i,j = 1,..., N. Many interesting jobs are composed by indepen- 
dent tasks (e.g. serving parallel machines, simple assembly jobs). 

We aim at designing a task assignment and dynamic scheduling architecture 
that: 


— Builds optimal nominal task schedules for the human and for the robot, 
i.e. two task schedules such that, considering the nominal execution times, 
minimizes the job execution time and maximizes the parallelism between 
human and robot (i.e. minimizes idle time) 

— Starting from the nominal task scheduling, reschedules the robot tasks ac- 
cording to the effective execution time detected by the monitoring unit and 
the decisions taken by the human and the robot for task swapping. The 
rescheduling aims at minimizing the execution time. 


Continuously and automatically changing the order of the tasks assigned to 
the human can lead to confusion and poor efficiency of the operator [25]. Thus, 
we have chosen to reschedule online only the list of tasks assigned to the robot. 
The list of tasks assigned to the human changes only when necessary, namely 
when the human decides to execute a task assigned to the robot or when the 
robot cannot execute a task and asks for the help of the human. In this case, 
the lists of tasks assigned to the human is changed minimally. 

The proposed task assignment and dynamic scheduling strategy can be repre- 
sented by the architecture in Fig.|1| where two main layers can be distinguished: 


1. Task Assignment. It is responsible of generating the initial nominal sched- 
ules for the robot and the human, based on the maximum parallelism crite- 
rion. 


'The choice of the specific technique for splitting a job into several tasks is out of 
the scope of this paper. Several strategies are available in the literature (see, e.g., 
for assembly tasks.) 
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Fig. 1. The overall architecture. The blue blocks represent the two layers. The yellow 
blocks, instead, symbolize the strategies implemented to provide richer information to 
the Dynamic Scheduler. The red blocks represent the two agents. 


2. Dynamic Scheduler. It is responsible of scheduling the tasks, taking into 
account the real execution time and the requests coming from the human 
and from the robot. 


3 Task Assignment Layer 
The role of this layer is to build nominal task schedules for the human and for 
the robot starting from the nominal execution times of each task T;. This is done 
by solving the following multi-objective Mixed Integer Linear Program: 

Ming,c oe (wre ri + wait) +e 

subject to 

(1) 
Crt @y,=1 We {l,...,N} 
N 
Vii tila)tai SC Vae A 


The terms wri, WąHi > 0 represent the weights for executing task T; on behalf 
of the robot and of the human, respectively. The boolean variables £pRi, £ Hi € 
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{0,1} are detecting whether T; is assigned or not to the robot or to the human, 
respectively; £ = (£ R1,- --, ZRN; ZH1;,---, ZHN) is the vector containing all the 
decision variables. Finally, t;(a) > 0 represents the nominal execution time of T; 
on behalf of agent a € A and c > 0 denotes the cycle time. 

The weight terms wr; and wy; are exploited for encoding the skills of the 
collaborative agents in the execution of each task. The better the agent is at 
executing a task, the lower the corresponding weight. Very high weights are 
exploited for communicating to the task assignment algorithm that an agent is 
unsuitable for the execution of a task. In these terms job quality information can 
be considered. The first constraint guarantees that each task is assigned either 
to the robot or to the human. The second constraint maximizes the parallelism 
between the human and the robot. In fact, since all the terms in the quantity 
to minimize are positive, the optimization problem would tend to choose c as 
small as possible and the lower bound for c is given by the last constraint and 
corresponds to the maximum parallelization of the activities of the human and 
of the robot. 

The outcome of the optimization problem are Sy and Sp, the set of tasks 
that have to be executed by the human and by the robot, respectively. The tasks 
in Sy and Sp are then organized in the increasing order of their indexes. This 
generates the nominal schedules, i.e. two ordered lists Ly and Lpr containing the 
tasks that have to be sequentially executed by each agent. 


4 Dynamic Scheduler 


Starting from the nominal schedules Ly and Lp, the goal of the dynamic sched- 
uler is to take explicitly into account the variability of the human robot collab- 
oration. When two humans are collaborating, if one gets slower the other tries 
to speed up its work in order to keep good performance of the team minimizing 
waiting times. Furthermore, difficulties are handled by communication. The most 
expert member of the team can decide to take some of the work or reorganize 
the work based on its experience. On the other side, when in trouble, the less 
experienced member of the team asks the expert member for some help. The 
dynamic scheduler aims at reproducing this kind of behavior in human-robot 
collaboration in order to create an effective and intuitive cooperation. 

This is achieved in two ways. First, by monitoring in real time the work of the 
human operator in order to estimate the real execution time and, if necessary, 
to reschedule the activities of the robot in order to avoid useless waiting times. 
Second, by enabling the human and the robot to communicate and take decisions 
about their activities through the scheduler. In particular, the robot allocates a 
task that it cannot execute to the human. The human can decide to execute the 
task that the robot is doing (because, e.g., from its experience, it feels that the 
robot is not executing the task in a proper way). Furthermore, the human can 
decide to re-allocate some its tasks to the robot. 

The dynamic scheduler is implemented according to the pseudo-code reported 


in Alg. 
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Algorithm 1 DynamicScheduler() 
1: Require: Ly,LR 

2: Endr, Endy + false 

3: Tr Lr(1); TH + Ly(1) 

4: while (Tr 49 and Ty #9) do 


5: Endr + monitorR(Tr) 

6: (Lr, Endy) + reschedule(T'z, Lr) 

7: My + readiz(), Mr + readr() 

8: (Endy, Endr, Ly, Lr) = communication(Mz, Mr, Lu, Lr) 
9: if Endy then Ty = next(Ty, Ly) 

10: end if 

11: if Endr then Tr = next(Tr, Lr) 

12: end if 


13: end while 


The dynamic scheduler needs as an input the nominal tasks schedules Dy 
and Lr (Line[1). It immediately sets to false the two variables Endp and Endy 
that identify when the task currently associated to the robot and to the human 
have been concluded (Line P) and it assigns the first tasks of Lr and of Ly to 
the human and to the robot (Line[3). The algorithm starts to loop until no more 
tasks are available for the robot and for the human (Line (4p. In the loop, the 
scheduler first checks if the robot finished the assigned task (Line [5p. The func- 
tion monitoR(Tpr) is application dependent and it verifies (e.g. by a camera, 
by a timeout procedure) if the task Tr has been successfully accomplished. If 
this is the case, the monitorR function returns a true value, false otherwise. 
Furthermore, if the robot cannot succeed to execute Tr (e.g. the task cannot 
be achieved after a predefined amount of time), the function monitorR gen- 
erates a delegate message Mp and sends it to the scheduler. The monitorR() 
function can be implemented using standard procedures, available for robotic 
applications (see e.g. [26)). The human execution time is monitored and the list 
of tasks assigned to the robot is adapted in order to maximize in real time the 
parallelism. (Line [6}. Then the messages generated by the human and the robot 
are considered for task swapping (Line [5}. Finally, the algorithm checks if the 
task assigned to the human and to the robot are over and, if it is the case, it 
assigns them the next task in the list (Lines [o] mah. The function next(T, L) 
returns the task after T in the list L and, if T is the last task of the list, it 
returns Ø. When both Tr and Ty are empty, then the job is over. 

The rescheduling algorithm is represented in Alg. 

The algorithm requires as input the tasks Ty and Tp that are currently 
assigned to the human and to the robot and the current schedule Lpr of the 
tasks for the robot (Line (i). The activity of the human is monitored and the 
remaining time tres for the accomplishment of Ty is estimated (Line [2). The 
procedure monitorH can be implemented using several strategies available in 
the literature as, e.g., 24). If tres is greater than the time necessary for the 
execution of task Tz, then some more tasks in Lr may be executed in parallel 
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Algorithm 2 Reschedule() 


1: Require: Ty,Tr, Lr 

2: tres — monitorH (Ty) 

3: if tres > tr(R) then 

4: (pLpr, fLr) <— split(Tr, Lr) 

5: flR < fill(fLr, tres — tr(R)) 

6: Lr + concat(pLr.LR, flr/ fle) 
7: end if 
8: Endy + nottres 
9: return(Lr, Endy) 


with Ty and, therefore, the rescheduling procedure starts (Line |3). First, the 
list Lpr is split into two sub-lists: pr contains Tp and the previous tasks while 
fLpr contains the other tasks to be executed Line [4] Then, from fLp, a sublist 
L'h of the tasks that can be executed in the extra time available tres — tr(R) is 
generated Line |5| The list L7% can be generated using any version of the well- 
known knapsack algorithm or more simple, job dependent, search techniques. 
Finally a new schedule for the robot tasks is built by concatenating pLpr, with 
the list of the rescheduled tasks f L'R and with the list of the remaining tasks to 
be executed Line [6| After the rescheduling, the Endy is set to true if Ty is over 
or to false otherwise (Line[8) and, finally, the new schedule Dr and the Endy 
variable are returned. 

During the execution of the job the human and the robot can generate mes- 
sages in order to communicate to the Dynamic Scheduler the intention or need 
to swap their tasks. A detailed representation of this communication layout is 
shown in Fig.|2} In particular, the message Mp sent by the robot can be either 
empty or containing the value “delegate Tr” and it is generated by the mon- 
itorR function when the robot cannot succeed in executing the assigned task. 
The message My can be either empty or it can assume two values: “reassign 
Treas’ or “delegate Tye;”. The first message is generated when the human de- 
cides to execute the task that the robot is executing (because, e.g., the robot is 
not doing the assigned work properly or in the best way). The second message 
is generated when the human decides to delegate some of the tasks in Ly to the 
robot. This message has an argument, that specifies the task to be delegated. 
The human can enter the messages through a proper, job dependent, input in- 
terface. The messages generated by the human and by the robot are handled by 
Alg. 

The algorithm requires the messages generated by human and robot Mp and 
My and the current schedules Lr and Ly and the task Tp currently assigned 
to the robot (Line[1). The message My is the first to be handled in order to give 
priority to the decisions taken by the operator. If the human decides to execute 
a task that was initially assigned to robot (i.e. My = “reassign Treas”), both 
the end of the task variables are set to true. In this way, in Alg.|1} the robot and 
the human will be assigned a new task (Lines[3] and [4). If the reassigned task is 
the one that the robot is executing (Line[5), an homing task Thome is put as the 
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Fig. 2. Communication Layout. The dotted lines indicate the “reassign” message com- 
ing from the human and the consequent task scheduling. The dashed lines indicate the 
“delegate” message coming from the human with the following scheduling to the robot. 
Finally, the brown lines indicate the “delegate” message communicated by the robot. 


next task in the robot schedule (Line|6). The task T)-eqs is then deleted from Lr 
‘nel while the task Tr is pushed in the first position of the human schedule 
(Line]9). In this way, the Alg. [1] will allocate Thome, that will send the robot to 
a safe home position, and Tr, the task the human has decided to execute, as 
the next tasks for the robot and the human. If the human decides to allocate a 
task Tae: € Ly (ie. My = “delegate Tye”) and the task is executable by the 
robot (Line[10), then Tye: is deleted from Ly (Line|11) and transferred into the 
task schedule of the robot (Line[12). If the robot detects that it cannot fulfill the 
assigned task (i.e. Mr = “delegate” ) and if the task it is trying to accomplish is 
executable by the human operator (Line 14), then the end of the task variable 
for the robot is set to false to force Alg.|1}to allocate the next task to the robot. 
Tr is deleted from the robot schedule Lr (Line|16) and inserted in the schedule 
of the human (Line [18). Furthermore, an homing mission Thome is added as 
the next task for the robot. Finally the procedure returns the updated end of 
task variables and schedules (Line|20). The procedure exRobot and exHuman 
exploit prior information about the job and the tasks (e.g. the weights wr; and 
WH; in (1)) to detect if a task can be executed by the robot or by the human. 


5 Experiments 


The proposed two-layers framework has been experimentally validated in a col- 
laborative assembly job consisting of storing 4 plastic shapes, fixing 3 little PCBs 
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Algorithm 3 Communication() 
1: Require: My, Mr, Tr, Lu,Lr 
2: if My = reassign(Treas) then 
3: Endr + true 
4 Endy < true 
5: if Treas = Tr then 
6: LR <= push(Thome, Lp) 
7: 

8 


end if 
: Lr + delete(Treas, Lr) 
9: Ly + push(Treas, La) 
10: else if My = delegate(Tae:) and exRobot(Ty-1) then 
11: Ly + delete(Ty.1, Li) 
12: LR q= push (Taer, Lp) 


13: end if 
14: if Mr = delegate(Tr) and exHuman(Tpr) then 
15: Endr < true 


16: LR $m delete(Tr, Lr) 

17: Lpr + push(Thome, LR) 
18: DRe push(Tp, Ly) 

19: end if 

20: return Endy, Endr, Lu, Lr 


and positioning a big PCB and a wooden bar. The human operator cooperated 
with a Kuka LWR 4+, a 7-DoF collaborative robot, at whose end-effector a 3D 
printed tool was attached. This tool allowed picking the objects using magnets. 
To monitor the human task execution we used a Kinect V2 RGB-D Camera 
with the official APIs for the skeleton tracking, while to evaluate the remaining 
task time we implemented the OE-DTW algorithm (see [28)) to the operator 
wrists positions. This algorithm compares an incomplete input time series with 
a reference one, returning as output the fraction of the reference series that cor- 
responds to the input. This fraction corresponds to the percentage of completion 
of the task %compi and it is used to estimate the remaining time as following: 


tres = (1 = Pcompi)ti(H) (2) 


For the communication interface, instead, we developed an HMI that allowed 
the operator to generate the wanted messages with the keyboard of the computer. 
A complete setup of the experiment is shown in Fig. 

All the software components were developed using ROS Melodic Morenia 
meta-operating system and they ran on a Intel(R) Core(TM) i7-4700MQ CPU @ 
2.40GHz with Ubuntu 18.04. The optimization problem was implemented using 
JuMP [29], a modelling language embedded in Julia programming language, and 
solved with Cbc solver [80]. 

We divided the collaborative assembly job in 11 different tasks, (Ti, ..., T11). 
Each task consists of one or more actions, this is due to create a more natural 
collaboration and communication. In fact, sometimes it could be more efficient 
to reassign and delegate a sequence of actions instead of a single one. A detailed 
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Fig. 3. Setup of the experiment. The two images show all the equipment used during 
the experiments. Moreover it is possible to note the presence of two specular shape 
selectors. The black one is fixed and it represents the area where the shapes were 
picked. The white selector, instead, is the one placed and screwed during the assembly 
job. It represents the area where the shapes were placed. 


description of the actions to be performed for each task is provided in Tab. [I] The 
collaboration of the two agents started after the tasks Tı and Tz were completed. 

In the experiments we considered as input for the Task Assignment Layer (see 
Sec. B) the data in Tab. [2] The weights wp; and wy; have been estimated taking 
into account the distance in the shared workspace between the task components 
and the agent and the intrinsic capability of each agent to accomplish the task 
(e.g. placing the screws is more difficult for the robot, because it requires a high 
precision). It is important to notice that all the tasks durations are less than 
equal to one. This is due to the fact that all the durations were normalized 
respect to the maximum nominal one, which is trobot,e = 40 sec. The nominal 
durations were precalculated by measuring many times the time required for 
each agent to perform the tasks and taking the average value. 

The solution of the optimization problem was a first nominal schedule that 
minimize the cost function, which was composed by the following two lists: 


— Ly = (1,2,3,4,5,6) 
— Lr = (7,8,9, 10, 11) 


Starting from the output of the Task Assignment Layer, the Dynamic Sched- 
uler was then initialized and tested in the collaborative assembly scenario. A 
complete video of the experiments is attached?| The first part of the video is 
dedicated to the first experiment where the two agents, the human and the 
robot, execute exactly the expected tasks, namely the ”nominal schedule”. Ini- 
tially the robot is in idle, because the tasks T, and To are preparatory for the 


“https: //youtu.be/48pH6MpSytM 
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Task Index 


Table 1. Tasks Action 


Description 


1 


Pick&Place of a shape sorter. 
Pick&Place of the screws. 


Screwing of the shape sorter. 


Pick&Place of the first PCB. 
Pick&Place of the screws. 
Screwing. 


Pick&Place of the second PCB. 
Pick&Place of the screws. 
Screwing. 


ol 


Pick&Place of the third PCB. 
Pick&Place of the screws. 
Screwing. 


Pick&Place of the bigger PCB. 


Pick&Place of a cross shape. 


Pick&Place of a circular shape. 


Pick&Place of a U shape. 


Pick&Place of a square shape. 
Pick&Place of a wooden bar. 


Table 2. Task Assignment Data 


Task Index WRi t;(R) WHi ti(H) 
1 0.6 0.500 0.14 0.375 
2 0.4 0.375 0.06 0.250 
3 0.8 0.875 0.2 0.625 
4 0.8 0.875 0.2 0.625 
5 0.8 0.875 0.2 0.625 
6 0.9 1.000 0.1 0.375 
T 0.3 0.350 0.5 0.250 
8 0.3 0.350 0.5 0.250 
9 0.3 0.350 0.5 0.250 
10 0.3 0.350 0.5 0.250 
11 0.2 0.250 0.9 0.750 
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collaborative job, which is not started yet. After the human confirms the com- 
pletion of that task, the Dynamic Scheduler allows the robot to start executing 
the parallel tasks. During the execution of T3 the monitoring strategy is acti- 
vated and the framework starts to estimate the remaining time (Tres). After the 
robot executes Ty and Tg, the estimated T;.., becomes lower than the required 
time of all the other robot tasks (see Alg. Line [3) and no other tasks can be 
executed by the robot, e.g. fL = 0 (see Alg. Line[5). At this point, the hu- 
man slows down while performing the screwing action, causing an increment in 
the estimated remaining time. Thanks to the adopted rescheduling strategy the 
vector of the rescheduled tasks is filled with the other available parallel tasks, 
flip = {Ts, To}, and the robot starts executing the first one. 

In the second experiment, the communication strategy is exploited. The oper- 
ator starts to execute the nominal schedule concluding T. However after placing 
the screws, he sends a message “delegate T>” to the Dynamic Scheduler. The 
start of the collaborative job is anticipated and the robot executes the screwing 
action while the operator is free to proceed his schedule. Please note that during 
the experiments it was not possible to execute e real screwing, for this reason we 
simulate the task by placing the robot over the screws and applying a rotation. 
After a while, thanks to his the great expertise, the operator realizes that the 
robot will place in a wrong way the “U” shape, which is the task Ty. For this rea- 
son, the operator executes Ty instead of the robot and communicate the message 
“reassign Tg” to the Dynamic Scheduler. At this point the Dynamic Scheduler 
deletes that task from the schedule of the robot and the robot executes the next 
one, Tio. 


6 Conclusion and Future Works 


In this paper we propose a two-layers framework for task assignment and schedul- 
ing for collaborative cells. Exploiting the tracking of the human body, the frame- 
work takes into account real execution time of the human to adopt an effective 
dynamic rescheduling strategy. Moreover, in order to take advantage of the ex- 
pertise of the operator and deal with possible errors, the presented framework 
allows the real-time communication between the human and the robot. The 
experimental evaluation shows the effectiveness of the framework and its appli- 
cability in a real industrial scenario, with parallel tasks involved. 

Future work aims at removing the assumption of independency of the tasks, 
that can be quite restrictive for some application. Furthermore, in order to fur- 
ther improve the cooperation between the human operator and the robot, we 
will investigate how the rescheduling and the communication system affect the 
human cognitive workload, taking more explicitly into account all the dimen- 
sions of job quality. This will lead to the possibility of optimize in real-time 
the industrial process, not only minimizing the idle times and maximizing the 
parallelism but also improving the well-being of the human operators. 
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